Computer programs often request several operations to be performed in response to a single event. For example, when a user of a photo management application requests to view a page of 100 photos, the photo management application will typically respond by requesting execution of 100 image retrieval operations.
Frequently, computing resources are not available to perform all of the desired operations at once. For example, a device with a single core processor would not be able to perform the 100 image retrieval operations in parallel. Consequently, it is common in these situations for computer programs to respond to events by placing multiple operation requests in a queue. Once the operation requests have been placed in a queue, the computing device executes the operations in a sequence that is dictated by the position of the operation requests within the queue, as processing resources become available.
In a system that queues operation requests, the order in which the requests are queued may have an enormous effect on the user experience. For example, assume that a user is using a device that is only able to display ten images at a time. If the retrieval requests for the first ten images of the 100 photo page are placed in the queue ahead of the other retrieval requests for the other 90 images, then the user will see the screen quickly fill with images. On the other hand, if the retrieval requests for the first ten images are placed in the queue after the retrieval requests for the other 90 images, a long time may lapse before the user sees any images at all.
To ensure the best user experience, computer programs can assign priorities to the operations that they request. These priorities may be passed to the process that is responsible for managing the queue (the “queue management unit”) along with the operation requests to which the priorities correspond. For example, the photo management application may indicate that the first ten photos are “high” priority, while the remaining 90 photos are “low” priority. Based on the priority information, the queue management unit would place the high priority operation requests for the first 10 photos ahead of the low priority operation requests for the other 90 photos
Unfortunately, even when priorities are provided to the queue management unit, inefficiencies may occur if there are significant delays between when any operation request is placed in the queue, and when the operation request is executed. The length of such delays is based on the number of operations in the queue, the resource requirements of the operation, and the resources available on the device. For example, if 100 over-the-network image retrieval requests are placed in a queue, the images are extremely high resolution, and the network connection speed is very slow, then there will be a long delay between when the requests are placed in the queue and when the 100th photo is finally downloaded and displayed.
When significant delays occur between when an operation request is queued for execution and when the operation is executed, the order of the operation requests within the queue may cease to reflect the desires of a user. For example, assume that after the 100 image retrieval operations have been queued, the user scrolls to the bottom of the 100 image page. The requests for the last 10 images may be at the very end of the queue (since they would have had the lowest priority at the time that the operation requests were submitted by the photo management application to the queue management unit). Consequently, the user may have to wait for the preceding 90 photos to be retrieved before the user begins to see the final 10 images.
One approach to avoiding the negative user experiences caused by such delays is to decrease the number of requests that computer programs send to the queue management unit at any given time. For example, rather than send operation requests to retrieve 100 images on a page, the photo management application may split the 100 image page into ten 10 image pages. Using this approach, each request will only be for 10 images at a time. Thus, if the user requests the last 10 images, retrieval of those 10 images will be requested without placing requests for the 90 other images into the queue.
Unfortunately, changing a program to reduce the number of operation requests that are submitted to the queue management unit also has drawbacks. For example, some users may want to have 100 images per page, rather than ten 10 image pages. Further, it is often preferable for a requesting program to submit a large number of operation requests to the queue management system so that, after performing the high priority operations, the device can start on the low priority operations rather than simply going idle.
The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section.